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:: introduction :: 


The term stratigraphy refers to the recording of layers or ‘strata’ primarily in the discipline 
of geology and archaeology. Over the previous year Paterson has adapted the metaphor of 
stratigraphy to form a conceptual platform for the design, data-structure and communication of 
content - in this case sound samples or spatialised sources - in a changing spatial-temporal context. 
The following text will elaborate this approach, describe the particular objectives of the example 
presented, and further detail it’s content design. 


:: approach :: 


The basis of this approach is formed on the premise that the interactor of a LISTEN 
environment occupies a dynamically changing soundscape, responsive to changes in the spatial 
movement of the user, and also to the passage of time or duration of sound. Hence, a stratigraphical 
approach to this space aimed to assist the author of the environment to organise sound content into 
units of narrative potential, according to the spatial and temporal relationships between them. 

Borrowing terminology and conceptual structure from archaeological data recording methods: Units 
or loci have been defined with relative relationships to other loci in a given area, for example 
directly before, after or contemporary*. According to these basic relationships, each locus may be 
construed to exist in different layers or strata that represent the ‘present-tense’ of the soundscape 
at a given moment in its occupation. 

The system presumed that the strata were replaced consecutively mirroring the passage of time 
according to defined durations, or that they may be changed consequentially due to the interactor 
triggering change of the ‘present-tense’ with their agency. As a larger component of content 
presentation, the strata of loci could be bundled into a phase or period, allowing greater authorship 
of the linear and non-linear (i.e. user agency activated) change. 


Of course less-discrete temporal relations exist, concurrent with sound design, such as overlap, within, etc. but 
these were not implemented in the stratigraphic example presented. 
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Within the changing relation-structure described previous, the binding of content presentation - 
a soundscape created from sound samples - to the user’s interaction in the LISTEN environment, 
may be elaborated as follows: 

Each locus represents a field of agency (spatial area, with dimensions of size and position), which 
activates spatialised sound sources contained within, or associated to the locus entered / activated 
by the interactor. Using the notion that a locus may be permeable* - allowing sound to pass 
into it - variable indexes are configured, either during the authoring process, or in response to 
occupation, so that upon activation the locus ‘gathers’ spatialised or ambient sound sources to play 
from surrounding loci, both spatially and temporally. For example, upon activation, a locus which is 
permeable with its contemporaries and relations previous, gathers sound sources from it’s contem- 
poraries and it’s before-relations, that due to a scalar permeability-value of, say, 2, exist within a 
spatial field twice its own. Therefore, those gathered - the sound sources & samples associated to 
that locus - that ‘field of agency’ - are what can be heard when activated by the interactor. 

:: objectives of workshop example :: 


The main purpose of the demonstration example for the Second LISTEN Expert Workshop 
was to explore the potential of the stratigraphical metaphor for authoring poetic, spoken-word sound 
content. Hence, the activity expressed the personal creative interest of Paterson, and aims to 
anticipate authorship issues for the creative involvement of poet Oswald Egger in the final LISTEN 
exhibit, to be presented at the Kunstmuseum Bonn in June 2003. 

Such ambition involved consideration of the necessity to structurally manage, and author for, 
both linear and so-called non-linear change, affected by the user’s movement during their 
occupation of the LISTEN space. How may the author of the textual content accommodate 
multiple possible locations in the narrative presentation, over certain durations of time? To what 
effect does authorship ‘control’ assist or hinder the experience? And to what extent does the 
influence of ‘granularity-in-change’ effect interpretation? 

Further, the example aimed to further dialogue regarding the remote, interactive and ‘non-bodied’ 
performance of the spoken word, and the semantic/active potential of words ‘performed’ in response 
to the agency of the listener. Lastly, through presenting the metaphor of stratigraphy as a 


* This quality of the locus continues the metaphorical association of the spatio-temporal unit of agency to that of 
the excavated archaeological stratigraphical deposit, usually some form of natural material such as soil, mud, 
stones etc.: A variable of context is its porous nature, and so may assist to determine how much the of 
contained objects or artefacts may have actually originated from surrounding stratigraphical deposits, for example, 
temporally after. 
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conceptual platform for authoring immersive audio-augmented environments, Paterson aimed to 
question the usefulness, and seek the point of ‘breakage’, of (such) a metaphor to communicate the 
complex relationship between agency, space and time. 

As will be elaborated in the following section, Paterson used a high ‘granularity’ of structure to 
construct a spatio-temporal spoken-word environment to test the objectives above, and in general, 
the stratigraphical conception, using many loci, and thus many strata of change. 

Although, it was originally the intention to use spatialised sound sources in the ‘stratigraph’ example, 
which may exist outside the discrete agency boundaries of the locus, demonstrating the concept of 
permeability described previous, this was not possible. Due to technical/ quality constraints of using 
ORTF spatialised sound sources with the LISTEN sound server version-in-use during the workshop, 
and the lack of a dynamic sound-module allocation in Paterson’s example test code, there was a 
limitation to the presentation of the stratigraphical concept, as described above. 

Instead, as a compromise to the large number of loci in the stratigraph, less-computationally- 
demanding mono-file samples were used which were associated only to one locus (i.e. no loci had 
any dynamic permeability values). Admittedly, with experience, this focused the interpretation to 
a less subtle experience and enhanced the arguably limiting, discrete, aspects of the authoring 
structure. 

:: content design :: 


The demonstrated example was composed, using spoken-word samples, based upon a 
text written by Paterson, entitled ‘upper-interior’. The text, constructed from an evolving series of 
‘cut&pastes’ of the original sentence, takes the complete form of 12 similar sentences, broken into 
segments according to emphasis upon word and ‘sound-bite’ (appendix A). Worthy of remark, are 
the more or less ambiguous references to the activity and process that the interactor is involved, 
through immersion in an audio-augmented environment, and the ‘voice’ of the tracking system - 
the continuous stream of data generated with occupation within the system. Also, the intentional 
marking of the end of each sentence with ‘Stop’, ‘stop’, or a rhyming equivalent; evocative of a 
‘telegrammed’ message, repeated and consistently warped. 

The stratigraph is structured to reflect, with intent, authorship control of narrative sequence, 
placing the interactor in the uncontrollable passage of linear time, and thus the uncontrollable 
passage of sentence, leaving the user’s spatial agency as the arbiter of change in the content 
experienced. The sentences are arranged into parallel streams, where similar references or seg- 
ments are coordinated to co-exist in the same ‘present-tense’, the same strata, and leaving gaps 
where there was not a co-existence (appendix B). 
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It may be useful to conceptualise and visualise the sentences as a full stratigraph: a grouped block 
of segmented columns which collapses downwards at changes in the ‘present-tense’; a matrix of 
strata consisting of loci - contemporary, before and after - with gaps that represent an absence in 
content at a particular spatio-temporal moment (see figure 1 ). 

The listener’s spatial movement and, in relation, the time spent without movement, influences 
what they hear as time passes during immersion. Once triggered into activation, the stratigraphic 
strata changed according to linear time-scheduler at consistent intervals and following a short 
pause, looped through the strata sequence again (figure 2). Indeed the spoken-word structure was 
intended to reward patience, as to stand still would present the sentence in its, albeit paced, entirety. 
Further, the sound ‘installation’ aimed to explore an increasing sense or experience of structure 
while immersed in the loop. 


figure 1 . Stratigraph matrix of spatio-temporal ‘fields of agency’. 



stratum 
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figure 2. Plan & perspective Avango screenshots of changing strata during runtime. 

Loci reference tags, e.g. aO, relate to the textual segments in appendix B. 


stratum 


stratum 



















:: technology application :: 
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The LISTEN system demonstrated at the Expert Workshop in Paris was based upon three 
computer-networked components: 

* Real-time virtual ‘world-model’ graphical environment (see figure 2 for illustration) using the 
AVANGO VE distributable environment system (http://www.avango.org), running on high spec. 
LINUX workstation. 

* Advanced Real-Time optical tracking system (http://www.ar-tracking.de/info/ARTtrack1-en.pdf), 
which tracked a group of reflectors positioned on the user’s headphones, in a 6 x 6 metre area. 

* LISTEN Sound Server (MAX MSP environment), on a G4 Mac workstation. This component 
acted as a hub, receiving the location postion data, and rendering and delivering mono-file & 
ORTF spatialised sound files to user wearing the wireless, tracked headphones. 

All stratigraphical environment components were interfaced with the AVANGO system and scripted 
as Scheme (LISP-based) object-oriented classes. 


:: acknowledments :: 


The stratigraphy workshop and demonstration was developed and produced by 
Andrew Paterson during a 7-month period as visiting artist-researcher at the Fraunhofer Institute of 
Media Communication in Sankt Augustin, Germany, sub-contracted into the LISTEN project by the 
Kunstmuseum Bonn. For further information: http://listen.imk.fraunhofer.de 

The LISTEN system architecture utilised in this production was developed by Dr Gerhard Eckel, 
Joachim Goftmann, Matthias Haringer, Martin Voelkel (all IMK), Ramon Arroyo-Gonzales (sub- 
contract IMK), Olivier Warusfel & Olivier Delerue (IRCAM), Bemd Ruprechter (AKG Gmbh.). 

The 7-month residence period was supported by a Scottish Arts Council Lottery Training Award. 


Q Scottish 
Arts Council 

L DTTERY FUNDED 


KUNST 

MUSEUM 

BONN 



Fraunhofer |nstjtut 

Medienkommunikation 


^ A shared-cost RTD project in the Information Society Technologies (1ST) 

Mw Programme of the European Commission’s Fifth Framework Programme. 

Contract No.: I ST- 1999-20646. 



:: appendix A :: 


7 


‘upper-interior’ (original text) 


The complete essence | of what | may be seen | Within the black-red landscape | eyelids 
the upper interior | fully conceived | while you wait | Begins when this voice stops 

The complete essence | of what | can be felt | seen not without eyelids | The upper interior 
can you comprehend | while you pause - you hide nothing | And begins | covering of the full | Stop 

The complete | sense | of what is now heard | hidden not to the ears | stream of numbers 
do you understand | while i count - you know nothing | And ends with dot 

The complete shift in what | now heard is hidden | you compound the data | an alter ego 
while you | no | i know nothing | And so on the continuation of | the voice stops 

The partial shift | what is now seen is hidden | the black as interior exterior | it is listening 
Do you know that | And all that ends well | while you are | not | yes here | Now not 

The full but partial | what’s now known | can be found here | The exterior is interested 
while you do you cannot | you are listening | And all flows | stop 

The essence that | remember | remembers | found on the edge | the interior boundary-less 
without border | while you do not know | listening | And all that ends well | data | Stop 

The complete essence | Remember when i told you | you found | numbers | The geography 
is boundary-less | you do not know | as you are listening | And that is the beauty’s end | Stop 

The common essence | you hear when you see | You find laterally | no boundary to, to be 
The geography | You know you can listen | and you can choose not | And that is the end 

The complete sense you will need | the flow | For geography and history | holds memory 
real space I know nothing | And that is | the end | data | Stop 

The absolute sense | you cannot hear it | to comprehend | for geography and history 
folds memory | I know nothing | but the upper interior | Stop 

The absolute essence | of flows | Within the black-red landscape | geography behind eyelids 
the history interior | fully conceived | partial exterior | data streams end | when this voice stops 


appendix B :: 
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‘upper-interior’ (structured text into strata 0 - 11 , where 0 is first) 


aO The complete essence 
al of what 
a2 may be seen 

a3 Within the black-red landscape 

a4 eyelids 

a5 the upper interior 

a6 fully conceived 

a7 while you wait 

a8 

a9 

alO 

all Begins when this voice stops 

cO The complete 
cl sense 

c2 of what is now heard 
c3 hidden not to the ears 
c4 

c5 stream of numbers 
c6 do you understand 
c7 
c8 

c9 while I count - you know nothing 

clO And ends with dot 

ell 

eO The partial shift 

el what is now seen is hidden 

e2 

e3 the black as interior exterior 
e4 

e5 it is listening 

e6 Do you know that 

e7 And all that ends well 

e8 while you are 

e9 not 

elO yes here 

ell Now not 


bO The complete essence 
bl of what 
b2 can be felt 
b3 

b4 seen not without eyelids 

b5 The upper interior 

b6 can you comprehend 

b7 while you pause - you hide nothing 

b8 And begins 

b9 

bl 0 covering of the full 
bl 1 Stop 

dO The complete shift in what 
dl 

d2 now heard is hidden 
d3 

d4 you compound the data 
d5 an alter ego 
d6 

d7 while you 
d8 no 

d9 I know nothing 

dIO And so on the continuation of 

dll the voice stops 

fO The full but partial 
fl what’s now known 
f2 can be found here 
f3 The exterior is interested 
f4 

f5 while you do 

f6 you cannot 

f7 you are listening 

f8 And all flows 

f9 stop 

flO 

fll 
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‘upper-interior’ (continued) 


go 

The essence that 

gi 

Remember 

g2 

remembers 

g3 

found on the edge 

g4 

the interior 

g5 

boundary-less without border 

go 


g7 

while you do not know 

gs 

listening 

go 

And all that ends well 

g10 data 

gii 

Stop 

iO 

The common essence 

il 

you hear when you see 

i2 

You find laterally 

i3 


i4 

no boundary to, to be 

i5 

The geography 

i6 

You know you can listen 

i7 

and you can choose not 

i8 


i9 


ilO 

And that is the end 

ill 


kO 

The absolute sense 

kl 

you cannot hear it 

k2 

to comprehend 

k3 

for geography and history 

k4 

folds memory 

k5 


k6 

1 know nothing 

k7 

but the upper interior 

k8 


k9 


klO 


kl 1 

Stop 


hO The complete essence 
hi Remember when i told you 
h2 you found 
h3 numbers 
h4 

h5 The geography is boundary-less 
h6 you do not know 
h7 as you are listening 
h8 

h9 And that is the beauty’s end 
hlO Stop 
hi 1 


jO The complete sense you will need 
jl the flow 
}2 

j3 For geography and history 

j4 holds memory 

j5 real space I know nothing 

j6 And that is 

j7 the end 

j8 data 

j9 Stop 

jlO 

jll 

10 The absolute essence 

11 of flows 

12 Within the black-red landscape 

13 geography behind eyelids 

14 the history interior 

15 

16 fully conceived 

17 partial exterior 

18 
19 

110 data streams end 

111 when this voice stops 


